Version control for asynchronous distributed machine learning

ABSTRACT

The disclosed embodiments provide a system and method for performing version control for asynchronous distributed machine learning. During operation, the system transmits a first global version of a statistical model to a set of client computer systems. Next, the system obtains, from a first subset of the client computer systems, a first set of updates to the first global version. The system then merges the first set of updates into a second global version of the statistical model. Finally, the system transmits the second global version to the client computer systems asynchronously from receiving a second set of updates to the first and/or second global versions from a second subset of the client computer systems.

RELATED APPLICATION

The subject matter of this application is related to the subject matterin a co-pending non-provisional application by the same inventors as theinstant application and filed on the same day as the instantapplication, entitled “Regularized Model Adaptation for In-SessionRecommendations,” having serial number TO BE ASSIGNED, and filing dateTO BE ASSIGNED (Attorney Docket No. LI-P1572.LNK.US).

BACKGROUND

Field

The disclosed embodiments relate to distributed machine learning. Morespecifically, the disclosed embodiments relate to techniques forproviding version control in asynchronous distributed machine learning.

Related Art

Analytics may be used to discover trends, patterns, relationships,and/or other attributes related to large sets of complex,interconnected, and/or multidimensional data. In turn, the discoveredinformation may be used to gain insights and/or guide decisions and/oractions related to the data. For example, business analytics may be usedto assess past performance, guide business planning, and/or identifyactions that may improve future performance.

However, significant increases in the size of data sets have resulted indifficulties associated with collecting, storing, managing,transferring, sharing, analyzing, and/or visualizing the data in atimely manner. For example, conventional software tools and/or storagemechanisms may be unable to handle petabytes or exabytes of looselystructured data that is generated on a daily and/or continuous basisfrom multiple, heterogeneous sources. Instead, management and processingof “big data” may require massively parallel software running on a largenumber of physical servers and/or nodes, as well as synchronizationamong the servers and/or nodes.

Consequently, big data analytics may be facilitated by mechanisms forefficiently and/or effectively collecting, storing, managing,compressing, transferring, sharing, analyzing, and/or visualizing largedata sets.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a schematic of a system in accordance with the disclosedembodiments.

FIG. 2 shows an exemplary sequence of operations involved in performingasynchronous distributed machine learning in accordance with thedisclosed embodiments.

FIG. 3 shows the regularization of model adaptation for in-sessionrecommendations in accordance with the disclosed embodiments.

FIG. 4 shows a flowchart illustrating the process of operating a serverin an asynchronous distributed machine learning system in accordancewith the disclosed embodiments.

FIG. 5 shows a flowchart illustrating the process of operating a clientin an asynchronous distributed machine learning system in accordancewith the disclosed embodiments.

FIG. 6 shows a flowchart illustrating the process of performingregularized model adaptation for in-session recommendations inaccordance with the disclosed embodiments.

FIG. 7 shows a computer system in accordance with the disclosedembodiments.

In the figures, like reference numerals refer to the same figureelements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the embodiments, and is provided in the contextof a particular application and its requirements. Various modificationsto the disclosed embodiments will be readily apparent to those skilledin the art, and the general principles defined herein may be applied toother embodiments and applications without departing from the spirit andscope of the present disclosure. Thus, the present invention is notlimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features disclosed herein.

The data structures and code described in this detailed description aretypically stored on a computer-readable storage medium, which may be anydevice or medium that can store code and/or data for use by a computersystem. The computer-readable storage medium includes, but is notlimited to, volatile memory, non-volatile memory, magnetic and opticalstorage devices such as disk drives, magnetic tape, CDs (compact discs),DVDs (digital versatile discs or digital video discs), or other mediacapable of storing code and/or data now known or later developed.

The methods and processes described in the detailed description sectioncan be embodied as code and/or data, which can be stored in acomputer-readable storage medium as described above. When a computersystem reads and executes the code and/or data stored on thecomputer-readable storage medium, the computer system performs themethods and processes embodied as data structures and code and storedwithin the computer-readable storage medium.

Furthermore, methods and processes described herein can be included inhardware modules or apparatus. These modules or apparatus may include,but are not limited to, an application-specific integrated circuit(ASIC) chip, a field-programmable gate array (FPGA), a dedicated orshared processor that executes a particular software module or a pieceof code at a particular time, and/or other programmable-logic devicesnow known or later developed. When the hardware modules or apparatus areactivated, they perform the methods and processes included within them.

The disclosed embodiments provide a method, apparatus, and system forprocessing data. More specifically, the disclosed embodiments provide amethod, apparatus, and system for performing asynchronous distributedmachine learning. As shown in FIG. 1, a system for performing theasynchronous distributed machine learning may include a server 102 and anumber of clients (e.g., client 1 104, client y 106), which interactwith one another to produce multiple versions of a statistical model108. Each of these components is described in further detail below.

Statistical model 108 may be used to perform statistical inference,estimation, classification, clustering, personalization, recommendation,optimization, hypothesis testing, and/or other types of data analysis.For example, statistical model 108 may be a regression model, artificialneural network, support vector machine, decision tree, naïve Bayesclassifier, Bayesian network, hierarchical model, and/or ensemble model.The results of such analysis may be used to discover relationships,patterns, and/or trends in the data; gain insights from the input data;and/or guide decisions or actions related to the data. For example,statistical model 108 may be used to analyze input data related tousers, organizations, applications, websites, content, and/or othercategories. Statistical model 108 may then be used to output scores,provide recommendations, make predictions, manage relationships, and/orpersonalize user experiences based on the data.

In addition, statistical model 108 may be trained and/or adapted to newdata received on the clients. For example, the clients may be electronicdevices (e.g., personal computers, laptop computers, mobile phones,tablet computers, portable media players, digital cameras, etc.) thatproduce updates 114-116 to statistical model 108 based on user feedbackfrom users of the clients. Alternatively, the clients may generateupdates 114-116 to statistical model 108 in a distributed fashion ondifferent subsets of training data from server 102 and/or anothercentralized data source.

Each update may represent a local version of statistical model 108 thatis adapted to input data on the corresponding client. In addition, theclients may produce updates 114-116 from global versions (e.g., globalversion 1 110, global version x 112) of statistical model 108. Forexample, a client may periodically receive a latest global version ofstatistical model 108 from server 102. The client may then generate anupdate to the global version by providing user feedback and/or otherinput data as training data to the global version.

In turn, the clients may transmit updates 114-116 to server 102, andserver 102 may merge updates 114-116 into subsequent global versions ofstatistical model 108. After a new global version of statistical model108 is created, server 102 may transmit the new global version to theclients to propagate updates 114-116 included in the new global versionto the clients. The clients may then produce additional updates from thenew global version and provide the additional updates to server 102 forsubsequent generation of additional global versions of statistical model108.

Thus, statistical model 108 may be iteratively trained through thebidirectional transmission of data from server 102 to the clients andfrom the clients to server 102. Moreover, alternating the creation ofglobal versions of statistical model 108 from input data aggregated frommultiple clients with the updating of the global versions on the clientsmay prevent overfitting of statistical model 108 to input data onindividual clients.

In one or more embodiments, local versions of statistical model 108 areproduced on the clients to personalize statistical model 108 to users ofthe clients. More specifically, the clients may obtain a global versionof statistical model 108 from server 102. Each client may then updatethe global version in real-time based on user input or feedback from auser of the client. As a result, the local version of statistical model108 on the client may be adapted to the user's behavior or preferencesas the user interacts with statistical model 108. For example, theclient may track the user's searches, clicks, views, text input,conversions, and/or other feedback during a user session with a jobsearch tool. As each piece of feedback is received from the user, theclient may provide the feedback as training data for statistical model108 to customize the output of statistical model 108 to the user'scurrent job search activity. Consequently, the client may generaterecommendations of job listings based on aggregated training data usedto produce the global version, as well as the user's input during thecurrent session with the job search tool. Adapting statistical models toin-session user behavior is described in further detail below withrespect to FIG. 3.

In one or more embodiments, server 102 and the clients performasynchronous distributed machine learning, in which barriers or locksfor synchronizing the updating of statistical model 108 are fullyremoved. For example, server 102 may update statistical model 108 byproducing global versions of statistical model 108 and transmitting theglobal versions to the clients independently from receiving updates114-116 to the global versions from the clients. Since updates to theglobal versions are not affected by variations in the processing speed,computational power, and/or network delay of individual clients,statistical model 108 may be updated faster than distributed machinelearning techniques that include barriers or locks for synchronizingstatistical model updates.

More specifically, the system of FIG. 1 may use automatic versioncontrol to perform asynchronous distributed machine learning. As shownin FIG. 1, a version-management apparatus 118 in server 102 may trackglobal versions (e.g., global version 1 110, global version x 112) ofstatistical model 108 using a set of version identifiers (e.g., versionidentifier 1 122, version identifier x 124). Each version identifier mayrepresent a given global version of statistical model 108, which iscreated by a merging apparatus 118 that merges a subset of updates114-116 from the clients into one or more previous global versions ofstatistical model 108.

To track the subset of updates that have been merged into each globalversion, the corresponding version identifier may be generated from aset of update identifiers (e.g., update identifiers 1 118, updateidentifiers x 120) for the subset of updates. For example, each updateidentifier may specify the client from which the corresponding updatewas received, as well as the global version of statistical model 108used to produce the update. Version-management apparatus 118 mayconcatenate, hash, and/or otherwise combine update identifiers for agiven subset of updates 114-116 into the version identifier for theglobal version that will be produced from the updates. In turn, mergingapparatus 118 may use the version identifiers to ensure that all updates114-116 from the clients have been merged into the global versions ofstatistical model 108 while avoiding merging of each update more thanonce into the global versions. Thus, by tracking updates 114-116 tostatistical model 108 in version identifiers for the global versions,version-management apparatus 118 may allow the global versions to begenerated without synchronization barriers associated with updates114-116.

The asynchronous distributed machine learning performed by the system ofFIG. 1 may be defined as an optimization problem using the followingequations:

$\begin{matrix}{{\min\limits_{z,{\{{w_{1}\mspace{14mu} \ldots \mspace{14mu} w_{M}}\}}}{\frac{1}{2}{z}^{2}}} + {\sum\limits_{i = 1}^{M}\left( {{f_{i}\left( w_{i} \right)} + {\frac{\gamma}{2}{{w_{i} - z}}_{2}^{2}}} \right)}} & (1) \\{{f_{i}\left( w_{i} \right)} = {\sum\limits_{j = 1}^{N_{i}}{l\left( {x_{ij},{y_{ij};w_{i}}} \right)}}} & (2)\end{matrix}$

In particular, Equation 1 may represent a general formalization of theoptimization problem, where w_(i) is a local version of statisticalmodel 108 on a client (e.g., client 1 104, client y 106), the clientsare indexed from 1 to M, and z is a global version of statistical model108. Equation 2 represents a use of statistical model 108 in aclassification task, where l is a convex loss function, x_(ij) is aninput feature vector, and y_(ij) is the target to predict.

Equation 1 may be solved using a synchronous alternating directionmethod of multipliers (ADMM) technique. More specifically, localconsensus variables q_(i) may be added to Equation 1 to obtain thefollowing:

$\begin{matrix}{{{\min\limits_{z,q,{\{ w_{i}\}}}{\sum\limits_{i = 1}^{M}{f_{i}\left( w_{i} \right)}}} + {\frac{\gamma}{2}{{w_{i} - q_{i}}}^{2}} + {\frac{1}{2}{z}^{2}}}{{{{subject}\mspace{14mu} {to}\mspace{14mu} q_{i}} = z},{\forall{i \in \left\lbrack {1\mspace{14mu} \ldots \mspace{14mu} M} \right\rbrack}}}} & (3)\end{matrix}$

Equation 3 may also be converted into an augmented Lagrangian to obtainthe following:

$\begin{matrix}{{\mathcal{L}\left( {z,\left\{ {q_{i},w_{i},\lambda_{i}} \right\}} \right)} = {{\sum\limits_{i = 1}^{M}{f_{i}\left( w_{i} \right)}} + {\frac{\gamma}{2}{{w_{i} - q_{i}}}^{2}} + {\frac{\rho}{2}{{q_{i} - z}}^{2}} + {\lambda_{i}^{T}\left( {q_{i} - z} \right)} + {\frac{1}{2}{z}^{2}}}} & (4)\end{matrix}$

In Equation 4, λ represents an estimate of the Lagrange multiplier.

A synchronous ADMM solution may then be obtained using the following:

$\begin{matrix}{w_{i}^{t + 1},{q_{i}^{t + 1} = {{\arg \; {\min\limits_{q_{i},w_{i}}{f_{i}\left( q_{i} \right)}}} + {\frac{\gamma}{2}{{w_{i} - q_{i}}}^{2}} + {\frac{\rho}{2}{{q_{i} - z^{t}}}^{2}} + {\lambda_{i}^{t}\left( {q_{i} - z^{t}} \right)}}}} & (5) \\{\mspace{20mu} {z^{t + 1} = {{\arg \; {\min\limits_{z}{\frac{\rho}{2}{{q_{i}^{t + 1} - z^{t}}}^{2}}}} + {\lambda_{i}^{t}\left( {q_{i}^{t + 1} - z^{t}} \right)} + {\frac{1}{2}{z}^{2}}}}} & (6) \\{\mspace{20mu} {\lambda_{i}^{t + 1} = {\lambda_{i}^{t} + {\rho \left( {q_{i}^{t + 1} - z^{t + 1}} \right)}}}} & (7)\end{matrix}$

Equations 5, 6 and 7 may then be used to obtain the following updatingrules, where μ=/ρ:

$\begin{matrix}{{\forall i},{\mu_{i}^{t + 1} = {\mu_{i}^{t} + q_{i}^{t} - z^{t}}}} & (8) \\{{\forall i},{w_{i}^{t + 1} = {{\arg \; {\min\limits_{w}{f_{i}(w)}}} + {\frac{\rho \; \gamma}{2\left( {\rho + \gamma} \right)}{{w - z^{t} + \mu_{i}^{t + 1}}}^{2}}}}} & (9) \\{{\forall i},{q_{i}^{t + 1} = {{\frac{\rho}{\rho + \gamma}z^{t}} + {\frac{\gamma}{\rho + \gamma}w_{i}^{t + 1}} - {\frac{\rho}{\rho + \gamma}\mu_{i}^{t + 1}}}}} & (10) \\{z^{t + 1} = \frac{\sum_{i = 1}^{M}\left( {q_{i}^{t + 1} + \mu_{i}^{t + 1}} \right)}{M + \frac{1}{\rho}}} & (11)\end{matrix}$

In synchronous ADMM, Equations 8, 9 and 10 are performed in the clients,while Equation 11 is performed by server 102. Server 102 may wait foreach round of updates 114-116 to be submitted by the clients beforemerging updates 114-116 into a new global version of statistical model108. Thus, the rate at which server 102 generates global versions ofstatistical model 108 may be limited by the updating of statisticalmodel 108 on the slowest client.

An asynchronous ADMM may omit the synchronization barrier, thus allowingserver 102 to generate global versions of statistical model 108independently from the transmission of updates 114-116 to statisticalmodel 108 from the clients. In asynchronous ADMM, server 102 may use thefollowing updating rules to create new global versions of statisticalmodel 108:

$\begin{matrix}{{\overset{\sim}{z}}_{h{(j)}}^{t + 1} = {{\overset{\sim}{z}}_{h{(j)}}^{t} + {\frac{1}{M + \frac{1}{\rho}}\left( {q_{j}^{t} + \mu_{j}^{t} - q_{j}^{t - 1} - \mu_{j}^{t - 1}} \right)}}} & (12) \\{{\overset{\sim}{z}}_{k}^{t + 1} = {\frac{1}{B_{k}}{\sum\limits_{j \in B_{k}}{\overset{\sim}{z}}_{h{(j)}}^{t + 1}}}} & (13)\end{matrix}$

In the updating rules above, {tilde over (z)}_(h(j)) represents a globalversion of statistical model 108, where h(j) ε[1 . . . k−1] is themapping from a client j to a version identifier of a given globalversion from which an update on the client is produced, and B_(k) is theset of clients submitting updates (e.g., updates 114-116) for merginginto statistical model 108 at time k.

During asynchronous ADMM, server 102 may track the merging of updates114-116 into statistical model 108 using version identifiers (e.g.,version identifier 1 122, version identifier x 124) of global versionsof statistical model 108. For example, server 102 may iterativelygenerate new global versions of statistical model 108 using thefollowing, where K is the version number (e.g., version identifier) ofthe latest global version of statistical model 108 and Δ_(j) is anupdate (e.g., updates 114-116) to statistical model 108 from the jthclient:

input:  {Δ_(j), h(j)|j ∈ B}${\left. {{{for}\mspace{14mu} j} \in {B\mspace{14mu} {do}}} \middle| z_{h{(j)}} \right. = {z_{h{(j)}} + \frac{\Delta_{j}}{M + \frac{1}{\rho}}}};$endK = K + 1; ${z_{K} = {\frac{1}{B}{\sum_{j \in B}z_{h{(j)}}}}};$

Broadcast (z_(K), K);

The above steps may be performed without iteration locks so the globalversion is generated independently of updates 114-116 to statisticalmodel 108 from the clients. Once the global version is generated, theglobal version may be broadcast to the clients.

In turn, each client may generate updates 114-116 to statistical model108 using the following, where z_(K) is the latest global version ofstatistical model 108:

input : x, y, iAdd x, y to samples for client i;Update μ_(i), q_(i), w_(i) according to Equations 8, 9, and 10;Let μ_(i) ⁰ and q_(i) ⁰ be the original values;Δ=μ_(i)+q_(i)−μ_(i) ⁰−q_(i) ⁰;

Reduce (Δ_(i), k)

More specifically, client i may perform a local optimization by updatingintermediate variables μ_(i) and q_(i) and the local version ofstatistical model 108 w_(i) based on input data received at the clientand/or the output of the local version based on the input data. Next,the client may calculate the difference between the updated variablesand the previous versions of the variables. The client may then use areduce step in a distributed-processing technique such as MessagePassing

Interface (MPI) to produce an update (e.g., updates 114-116) tostatistical model 108 using the difference and the current globalversion of statistical model 108 from which the local version wasproduced.

The equations described above further include a number of parameters,which may be used to control and/or modify the behavior of server 102and/or the clients. First, an optimization parameter ρ may be used toadjust the rate of convergence of global versions of statistical model108, with a higher value for ρ resulting in a faster rate of convergenceto a less optimal result. Thus, ρ may be selected to balance theconvergence rate and the performance of statistical model 108.

Second, a regularization parameter γ may control the amount ofpersonalization of each local version of statistical model 108. When γis 0, the local version is fully personalized to the user of thecorresponding client and does not include any adaptation to the behaviorof other users. In other words, a value of γ for γ may result in thecreation of a local version of statistical model 108 that is completelyseparate from any global versions of statistical model 108. As γincreases, the personalization of the local version and convergence ofstatistical model 108 decrease, subject to an upper bound of ρ/2.

The regularization parameter γ may be adjusted to balance thegeneralization of statistical model 108 with the personalization ofstatistical model 108. For example, γ may initially be set to a largevalue to prevent overfitting of statistical model 108 to limitedavailable data for a user. As additional data is collected from the user(e.g., during one or more user sessions), γ may be decreased to adapt alocal version of statistical model 108 more to the behavior of the useron the corresponding client. If the behavior of the user also divergesfrom those of other users (e.g., based on aggregated input from theother users used to create global versions of statistical model 108), γmay continue to be decreased until γ reaches 0. In another example,different values of γ may be used with different sessions and/or otherintervals of interaction with each user, and the value of γ with thebest performance may be selected for the user.

While the asynchronous distributed machine learning technique has beendescribed with respect to the Equations above, those skilled in the artwill appreciate that the system of FIG. 1 may be used to process othertypes of problems. For example, the system of FIG. 1 may be applied toany type of convex optimization problem, including problems related topersonalization and/or recommendation.

By performing asynchronous distributed machine learning, the system ofFIG. 1 may improve training of statistical model 108 over conventionalmodel adaptation techniques that require synchronization barriers, useonly global versions of statistical models on all clients, or use onlypersonalized versions of a statistical model on each client. At the sametime, the optimization and regularization parameters may be used toadjust the convergence, performance, and/or personalization ofstatistical model 108 in a distributed environment with large numbers ofusers and/or multiple sets of training data. Consequently, the system ofFIG. 1 may provide large-scale personalization and/or recommendation viaasynchronous distributed machine learning.

Those skilled in the art will appreciate that the system of FIG. 1 maybe implemented in a variety of ways. First, the clients may execute onand/or include different types of software or hardware components. Forexample, the clients may include web applications, native applications,mobile applications, operating systems, device drivers, and/or othertypes of software. Similarly, the clients may execute on personalcomputers, laptop computers, workstations, portable electronic devices,and/or other types of network-enabled computing devices.

Second, one or more instances of server 102 may be used to aggregateupdates 114-116 from the clients into global versions of statisticalmodel 108. If multiple instances of server 102 exist, each instance maybe matched to a different subset of clients based on attributes of theinstance and the clients. For example, the network bandwidth, processorresources, and/or memory on a given instance of server 102 may bematched to a client based on the rate at which the client generatesupdates to statistical model 108, the importance of the client, therevenue associated with the client, and/or other metrics orcharacteristics associated with the client. Different global versions ofstatistical model 108 produced by the instances may then be merged intoa master global version through additional communication among theserver 102 instances.

Third, server 102 may use different version-control techniques to trackthe generation of global versions of statistical model 108 from updates114-116 received from the clients. For example, server 102 may hashupdate identifiers for updates to be merged into a global version ofstatistical model 108 into a version identifier for the global version.In another example, server 102 may generate version numbers for globalversions of statistical model 108 based on the number of updates mergedinto each global version, the types of clients from which the updateswere received, the time interval between global versions, and/or otherattributes associated with the global versions and/or updates 114-116.

Fourth, server 102 may use a number of techniques to aggregate updates114-116 into the global versions. As illustrated in the above equations,server 102 may merge a set of updates into a global version ofstatistical model 108 by averaging the contribution of each update.Alternatively, server 102 may merge the updates into the global versionaccording to the level of activity in each client, the recency of theupdates, the importance of the client, revenue associated with theclient, and/or other attributes of the client. Server 102 may furthertransmit portions of global versions of statistical model 108 to some orall of the clients to reduce network traffic between server 102 and theclients.

FIG. 2 shows an exemplary sequence of operations involved in performingasynchronous distributed machine learning in accordance with thedisclosed embodiments. As mentioned above, server 102 may use versioncontrol to produce global versions 210-216 of a statistical model (e.g.,statistical model 108 of FIG. 1) that are created and transmitted to aset of clients A 202, B 204, C 206, and D 208 independently of updates220-234 to global versions 210-216 from the clients.

In particular, server 102 may begin with a first global version 210 ofthe statistical model and assign it a version identifier of “1.0”.Server 102 may transmit global version 210 to clients A 202, B 204, andC 206. In turn, client A 202 may generate an update 220 to globalversion 210, and client C 206 may generate a separate update 222 toglobal version 210. Updates 220 and 222 may be transmitted to server 102and merged into a second global version 212 with a version identifier of“2.0A1C1.” To generate the version identifier of global version 212,server 102 may increment the version number (e.g., “2.0”) of thestatistical model, identify updates 220 and 222 as changes to globalversion 210 from clients A 202 and C 206, and append representations ofthe updates and the global version modified by the updates (e.g., “A1”and “C1”) to the version number.

Once global version 212 is created, server 102 may transmit globalversion 212 to clients C 206 and D 208, and client D 208 may generate anupdate 226 to global version 212. Server 102 may also receive an update224 to global version 210 from client B 204 after global version 212 isproduced. As a result, server 102 may merge updates 224 and 226 to twoseparate global versions 210, 212 of the statistical model into a thirdglobal version 214 with a version identifier of “3.0B1D2.” As with theversion identifier of global version 212, server 102 may generate theversion identifier for global version 214 by incrementing the versionnumber (e.g., “3.0”) of the statistical model, identifying updates 224and 226 as respective changes to global versions 210 and 212 fromclients B 204 and D 208, and append representations of the updates andthe global versions modified by the updates (e.g., “B1” and “D2”) to theversion number.

Server 102 may then transmit global version 214 to clients A 202 and B204. In turn, clients A 202 and B 204 may produce two separate updates230 and 232 to global version 214. After global version 214 is produced,client C 206 may transmit an update 228 to global version 212 to server102. Finally, server 102 may merge updates 228-232 into a fourth globalversion 216 with a version identifier of “4.0A3B3C2.” The versionidentifier may include a higher version number (e.g., “4.0”) thanprevious global versions 210-214, followed by representations of updates228-232 and version numbers of global versions 212-214 modified by theupdates (e.g., “A3,” “B3,” and “C2”).

By tracking the merging of updates 220-232 into global versions 210-216using version identifiers of global versions 210-216, server 102 maygenerate new global versions of the statistical model independently fromreceiving updates 220-232 to previous global versions from clients A202, B 204, C 206, and D 208. For example, server 102 may use theversion identifiers to ensure that updates that were previously mergedinto previous global versions are not merged into a new global versionof statistical model 108, and that updates that have been received fromthe clients but not merged into the previous global versions areincluded in the new global version. In other words, server 102 may usebidirectional model transmission with the clients and a version-controltechnique to perform asynchronous distributed machine learning.

FIG. 3 shows the regularization of model adaptation for in-sessionrecommendations in accordance with the disclosed embodiments. Morespecifically, FIG. 3 shows the personalization of a statistical model(e.g., statistical model 108) to a user during a user session 310 withthe user on a client (e.g., client 1 104, client y 106 of FIG. 1).First, the client may obtain a global version 302 of the statisticalmodel from a server, such as server 102 of FIG. 1. For example, theclient may download global version 302 from the server at the beginningof a first user session 310 with the user.

Next, the client may use the statistical model to interact with the userduring user session 310. For example, the client may implement one ormore features of a web application, native application, mobileapplication, and/or other type of application. During interaction withthe user in user session 310, the client may use global version 302 tooutput one or more recommendations 318 to the user. For example, theclient may recommend one or more job listings to the user during use ofa job search tool with an online professional network.

As shown in FIG. 3, the client may also use user feedback 314 from theuser to create a personalized version 306 of the statistical modelduring user session 310. Continuing with the above example, the clientmay create personalized version 306 from global version 302 at thebeginning of user session 310 and use the session identifier for usersession 310 as the update identifier for personalized version 306. Next,the client may track the user's clicks, views, searches, applications tojob listings, and/or other activity with the job search tool as userfeedback 314 from the user. Each piece of user feedback 314 may beprovided as training data that is used to create or update personalizedversion 306 during user session 310. For example, personalized version306 may be updated with each click and/or other type of input receivedfrom the user. In turn, the output of personalized version 306 may beadapted to the user's real-time behavior or preferences during usersession 310.

In one or more embodiments, training of personalized version 306 isaffected by the quality 308 of user feedback 314. Quality 308 may bebased on the context of and/or one or more attributes associated withuser feedback 314. For example, information submitted in one or moreform fields may represent higher quality user feedback 314 sincesignificant effort and/or time may be required to enter the informationin the form field(s). Conversely, a series of rapid clicks withoutadditional input from the user may represent lower quality user feedback314 because less time or effort is spent on generating the clicks and/orviewing content associated with the clicks, and thus the clicks are lesslikely to indicate user intent than user feedback 314 that requires moreuser interaction.

To train personalized version 306 based on quality 308 of user feedback314, each piece of user feedback 314 may be assigned a weight thatreflects quality 308. For example, higher quality user feedback 314 maybe weighted more than lower quality user feedback 314 so thatpersonalized version 306 is adapted more to the higher quality userfeedback 314 than the lower quality user feedback 314. After each pieceof user feedback 314 is assigned a weight, the weight may be provided asadditional training data to personalized version 306. Alternatively, theweight may be used to scale a value representing the corresponding userfeedback 314 before the value is inputted as training data topersonalized version 306.

Similarly, user feedback 314 may be labeled before user feedback 314 isprovided as training data to personalized version 306. For example, userfeedback 314 may be labeled as positive or negative feedback, withpositive feedback representing positive user actions (e.g., clicks,views, searches, conversions, likes, shares, upvotes, follows, etc.) andnegative feedback representing a lack of user action (e.g., non-clicksor ignores) or negative user actions (e.g., downvotes, dislikes, hides,unfollows, etc.). The labels may also include weights associated withquality 308. For example, strong positive labels for user feedback 314may be associated with longer viewing times and/or lengthier userinteraction, while weak positive labels for user feedback 314 may beassociated with short viewing times and/or an immediate return to aprevious screen of the application.

Once personalized version 306 is adapted from global version 302 basedon user feedback 314 during user session 310, personalized version 306may be used to output one or more additional recommendations 320 to theuser. More specifically, recommendations 320 from personalized version306 may be based on both user feedback 314 and previously outputtedrecommendations 318. First, recommendations 320 may be selected based ona similarity 330 to content associated with user feedback 314. Forexample, recommendations 320 may include job postings with highsimilarity 330 (e.g., similar job titles, industries, job descriptions,etc.) to other job postings with which the user has previouslyinteracted during user session 310. At the same time, becausepersonalized version 306 is adapted from global version 302 instead ofcreated only from relatively small amounts of user feedback 314 in usersession 310, overfitting of personalized version 306 to user feedback314 may be averted.

Second, recommendations 318 and user feedback 314 may be used bypersonalized version 306 to avoid including previously outputtedrecommendations 318 in newer recommendations 320. For example,personalized version 306 may track previously outputted recommendations318, 320 and user feedback 314 associated with the recommendations. Eachtime a recommendation is selected and shown to the user withoutreceiving positive user feedback 314 (e.g., a click) associated with therecommendation, the importance of the recommendation is discounted. Ifthe user continues to ignore the recommendation, the frequency withwhich the recommendation is selected and/or shown may continue todecrease until the recommendation is no longer outputted to the user.Instead, personalized version 306 may be used to output newrecommendations, which may be more relevant and/or interesting to theuser. In other words, personalized version 306 may be used to performimpression discounting of recommendations 318, 320 during user session310.

At the end of user session 310, the client may transmit an update 322containing a difference between personalized version 306 and globalversion 302 to the server. Once update 322 is provided to the server,the client may discard personalized version 306. In turn, the server mayuse update 322 and/or other updates to global version 302 or otherprevious global versions of the statistical model from other clients toproduce a new global version 304 of the statistical model. For example,the server may use version control to merge update 322 and/or the otherupdates into global version 304 asynchronously from receiving theupdates from the clients, as discussed above. The server may thentransmit global version 304 to the clients, and the clients may adaptglobal version 304 into personalized versions during individual usersessions with a set of users. As a result, the statistical model may becontinuously updated through the creation of per-session personalizedversions of the statistical model from global versions of thestatistical model on the clients and the subsequent merging of thepersonalized versions into new global versions of the statistical modelon the server.

In one or more embodiments, personalized version 306 and update 322 arecreated based on one or more parameters 324 associated with regularizedin-session adaptation of the statistical model. As described above,parameters 324 may include an optimization parameter and/or aregularization parameter. The optimization parameter may be used by theserver to adjust the rate of convergence of global versions 302, 304 ofthe statistical model.

The regularization parameter may be used by the client to control theamount of personalization of the statistical model to the user duringuser session 310. As discussed above, the regularization parameter maybe represented by γ in the above equations for performing asynchronousADMM. When γ is 0, personalized version 306 is fully adapted to theuser's in-session behavior and is completely separate from any globalversions of the statistical model. As γ increases, personalized version306 is affected more by global version 302.

The regularization parameter may be adapted to user feedback 314 and/orother user behavior or characteristics. For example, the regularizationparameter may initially be set to a large value to prevent overfittingof the statistical model to limited user feedback 314 during usersession 310. As additional user feedback 314 is collected, theregularization parameter may be decreased to adapt personalized version306 to the behavior of the user. If the behavior of the user divergesfrom that of other users (e.g., based on aggregated user feedback fromthe other users used to create global version 302), the regularizationparameter may continue to be decreased until the regularizationparameter reaches 0. In another example, different values of theregularization parameter may be used with different user sessions withthe user, and the value of the regularization parameter with the bestperformance may be selected for the user.

FIG. 4 shows a flowchart illustrating the process of operating a serverin an asynchronous distributed machine learning system in accordancewith the disclosed embodiments. In one or more embodiments, one or moreof the steps may be omitted, repeated, and/or performed in a differentorder. Accordingly, the specific arrangement of steps shown in FIG. 4should not be construed as limiting the scope of the embodiments.

Initially, the server is matched to a set of clients for distribution ofa statistical model based on attributes of the server and clients(operation 402). For example, the speed, amount of memory, and/ornetwork bandwidth of the server may be matched to the network traffic,popularity, and/or cost associated with the clients. Next, the servertransmits a first global version of the statistical model to the set ofclients (operation 404). For example, the server may merge a number oflocal versions of the statistical model from the clients into the firstglobal version of the statistical model. Alternatively, the serverand/or another component may generate the global version from apredefined set of training data. The server may then broadcast the firstglobal version to the clients for updating of the first global versionon the clients.

After the first global version is provided to the clients, the serverobtains a first set of updates to the first global version from a firstsubset of clients (operation 406). Each update may include a differencebetween a local version of the statistical model on the correspondingclient and the first global version. For example, the client may producethe update by training the first global version using data (e.g., userfeedback) on the client.

Next, the server merges the first set of updates into a second globalversion of the statistical model (operation 408). For example, theserver may generate the second global version as an average of the firstset of updates, as previously described with respect to the asynchronousADMM technique. Alternatively, the server may use a set of weightsassociated with the first subset of clients to merge the first set ofupdates into the second global version, so that some updates contributemore to the second global version than other updates. Each weight mayrepresent a recency, a level of activity, a popularity, a revenue,and/or another attribute associated with the corresponding update.

Similarly, the server associates the first set of updates with thesecond global version (operation 410). For example, the server mayobtain and/or generate update identifiers for the first set of updatesand/or the global versions used to produce the updates, and combine theupdate identifiers into a version identifier for the second globalversion. In other words, the server may use version control to track andmanage the asynchronous merging of updates from the clients into globalversions of the statistical model.

The server may then transmit the second global version to the clientsasynchronously from receiving a second set of updates to one or bothglobal versions from another subset of the clients (operation 412). Forexample, the server may generate and broadcast the second global versionwithout using any iteration barriers or locks to synchronize the updatesfrom the clients. To reduce network traffic to and/or from the clients,the server may also transmit a portion of the second global version(e.g., the difference between the second global version and the localversion on each client) in lieu of the entire second global version.

The server may then use the associated first set of updates to merge thesecond set of updates into a third global version of the statisticalmodel without merging the first set of updates into the third globalversion (operation 414). For example, the server may use the versionidentifier of the second global version to identify the first set ofupdates as already merged into the second global version. The server maythen generate the third global version by merging updates that have beenreceived since the first set of updates into the second global version.Finally, the server may transmit the third global version to the set ofclients (operation 416) for subsequent updating of the third globalversion on the clients, as described below with respect to FIG. 5. Aswith the second global version, the server may transmit one or moreportions of the third global version ti the clients in lieu of theentire third global version to reduce network traffic to the clients.

FIG. 5 shows a flowchart illustrating the process of operating a clientin an asynchronous distributed machine learning system in accordancewith the disclosed embodiments. In one or more embodiments, one or moreof the steps may be omitted, repeated, and/or performed in a differentorder. Accordingly, the specific arrangement of steps shown in FIG. 5should not be construed as limiting the scope of the embodiments.

The client initially obtains a first global version of a statisticalmodel from the server, as well as a regularization parameter (operation502) for the statistical model. The regularization parameter may controlthe level of personalization of the statistical model on the client.Next, the client uses the regularization parameter and data on theclient to produce a local version of the statistical model from thefirst global version (operation 504). For example, the client may outputone or more recommendations from the first global version and obtainuser feedback associated with the recommendation(s) from a user on theclient. The client may produce the local version from the first globalversion by providing the user feedback as training data to the firstglobal version and using the regularization parameter to adjust theamount by which the local version is affected by the training data. Theclient may then update the local version based on additional userfeedback and use the local version to generate additionalrecommendations during a user session with the user. In other words, theclient may use the local version to customize the output of thestatistical model to the user's preferences and/or behavior.

The client then transmits an update containing a difference between thelocal version and the first global version to the server asynchronouslyfrom receiving a second global version of the statistical model from theserver (operation 506). For example, the client may calculate the updatebased on the equations discussed above with respect to the asynchronousADMM technique. The client may then transmit the update at the end of auser session with a user, on a periodic basis, and/or according to othercriteria associated with updating the statistical model.

The client may also update the local version with the second globalversion based on the regularization parameter (operation 508). Forexample, the client may use the regularization parameter to combine thelocal version and the second global version during a subsequent usersession or other interval during which the statistical model is updatedon the client. Alternatively, the client may replace the local versionwith the second global version at the beginning of the subsequent usersession or model-training session if the local version is persisted onlyfor the duration of a single session.

The client may additionally update the regularization parameter based ondata on the client (operation 510). For example, the client may trackuser feedback that is used to update the statistical model during one ormore user sessions with the user. If the user feedback deviates fromexpected user behavior as captured by global versions of the statisticalmodel, the regularization parameter may be updated to increase thepersonalization of the local version to the user's behavior.Alternatively, the regularization parameter may be modified to reducethe personalization of the local version if the user's behavior issimilar to user behavior used to produce the global versions.

FIG. 6 shows a flowchart illustrating the process of performingregularized model adaptation for in-session recommendations inaccordance with the disclosed embodiments. In one or more embodiments,one or more of the steps may be omitted, repeated, and/or performed in adifferent order. Accordingly, the specific arrangement of steps shown inFIG. 6 should not be construed as limiting the scope of the embodiments.

Initially, a first global version of a statistical model is obtainedfrom a server (operation 602). The first global version may be obtainedby a client for performing the in-session recommendations. During afirst user session with a user, the client uses the first global versionto output one or more recommendations to the user (operation 604). Forexample, the client may use the first global version to recommend one ormore job listings to the user while the user interacts with a job searchtool on the client.

Next, the client uses the first global version and user feedback fromthe user to create a first personalized version of the statistical model(operation 606). For example, the client may generate the firstpersonalized version from the first global version by providing thefirst piece of user feedback as training data for the first globalversion. Once the personalized version is created from the first globalversion, the client may update the first personalized version inreal-time with each unit of user feedback received from the user. Duringcreation of the first personalized version, the client may determine oneor more weights for the user feedback based on the quality of the userfeedback. The client may then include the weight(s) and the userfeedback in training data for creating the first personalized versionfrom the global version. In turn, the weight(s) may allow the firstpersonalized version to be adapted more to higher quality user feedbackand less to lower quality user feedback.

The client then uses the first personalized version to output one ormore additional recommendations to the user (operation 608). Forexample, the client may use the first personalized version to discountthe selection of previously outputted recommendations that were ignoredby the user. In another example, the client may select additionalrecommendation(s) that are similar to content associated with the userfeedback. As a result, the client may adapt the statistical model to thein-session behavior of the user.

At the end of the first session, the client transmits an updatecontaining a difference between the first personalized version and thefirst global version to the server (operation 610). The update may thenbe used by the server to produce a second global version of thestatistical model, as discussed above.

After the second global version is produced and the first user sessionhas concluded, the client obtains the second global version from theserver (operation 612) and replaces the first global version with thesecond global version (operation 614). The client then uses the secondglobal version and additional user feedback from the user to create asecond personalized version of the statistical model during a seconduser session with the user (operation 616). Finally, the client uses thesecond personalized version to output one or more additionalrecommendations to the user (operation 618).

In other words, the client may create a separate personalized version ofthe statistical model from the latest global version during each usersession with the user. At the end of the user session, the client maytransmit an update representing the difference between the personalizedversion and the global version from which the personalized version wascreated to the server for use in producing a subsequent global versionof the statistical model by the server. Per-session user feedback at theclient may thus be used to adapt the statistical model to the user'sbehavior and generate subsequent global versions of the statisticalmodel.

FIG. 7 shows a computer system 700 in accordance with an embodiment.Computer system 700 may correspond to an apparatus that includes aprocessor 702, memory 704, storage 706, and/or other components found inelectronic computing devices. Processor 702 may support parallelprocessing and/or multi-threaded operation with other processors incomputer system 700. Computer system 700 may also include input/output(I/O) devices such as a keyboard 708, a mouse 710, and a display 712.

Computer system 700 may include functionality to execute variouscomponents of the present embodiments. In particular, computer system700 may include an operating system (not shown) that coordinates the useof hardware and software resources on computer system 700, as well asone or more applications that perform specialized tasks for the user. Toperform tasks for the user, applications may obtain the use of hardwareresources on computer system 700 from the operating system, as well asinteract with the user through a hardware and/or software frameworkprovided by the operating system.

In one or more embodiments, computer system 700 provides a system forperforming version control for asynchronous distributed machinelearning. The system may include a server that transmits a first globalversion of a statistical model to a set of client computer systems.Next, the server may obtain, from a first subset of the client computersystems, a first set of updates to the first global version. The servermay then merge the first set of updates into a second global version ofthe statistical model. Finally, the server may transmit the secondglobal version to the client computer systems asynchronously fromreceiving a second set of updates to the first and second globalversions from a second subset of the client computer systems.

Alternatively, computer system 700 may implement one of the clients. Theclient may obtain the first global version from the server. Next, theclient may update the first global version with data on the computersystem to produce a local version of the statistical model. The clientmay then transmit an update containing a difference between the localversion and the first global version to the server asynchronously fromreceiving a second global version of the statistical model from theserver.

The client may additionally perform regularized model adaptation forin-session recommendations. More specifically, the client may improve aperformance of the statistical model during a user session with a user.First, the client may use the first global version to output one or morerecommendations to the user. Next, the client may use the first globalversion and user feedback from the user to create a first personalizedversion of the statistical model. At the end of the first session, theclient may transmit the update containing the difference between thefirst personalized version and the first global version to the serverfor use in updating the first global version by the server.

In addition, one or more components of computer system 700 may beremotely located and connected to the other components over a network.

Portions of the present embodiments (e.g., server, clients, localversions, global versions, personalized versions, etc.) may also belocated on different nodes of a distributed system that implements theembodiments. For example, the present embodiments may be implementedusing a cloud computing system that performs asynchronous distributedmachine learning and/or in-session personalization based on updates to astatistical model from a set of remote clients.

The foregoing descriptions of various embodiments have been presentedonly for purposes of illustration and description. They are not intendedto be exhaustive or to limit the present invention to the formsdisclosed. Accordingly, many modifications and variations will beapparent to practitioners skilled in the art. Additionally, the abovedisclosure is not intended to limit the present invention.

What is claimed is:
 1. A method, comprising: transmitting, from a serverto a set of client computer systems, a first global version of astatistical model; obtaining, from a first subset of the client computersystems, a first set of updates to the first global version; merging, bythe server, the first set of updates into a second global version of thestatistical model; and transmitting the second global version to theclient computer systems asynchronously from receiving a second set ofupdates to one or more of the first and second global versions from asecond subset of the client computer systems.
 2. The method of claim 1,further comprising: associating the first set of updates with the secondglobal version; using the associated first set of updates to merge thesecond set of updates into a third global version of the statisticalmodel without merging the first set of updates into the third globalversion; and transmitting the third global version to the set of clientcomputer systems.
 3. The method of claim 2, wherein associating thefirst set of updates with the second global version comprises: combininga set of update identifiers for the first set of updates into a versionidentifier for the second global version.
 4. The method of claim 2,wherein using the associated first set of updates to merge the secondset of updates into the third global version without merging the firstset of updates into the third global version comprises: identifying theset of update identifiers in the version identifier for the secondglobal version; and omitting merging of the first set of updates intothe third global version based on the identified set of updateidentifiers.
 5. The method of claim 1, further comprising: matching theserver to the set of client computer systems for distribution of thestatistical model based on a first attribute of the server and a secondattribute of the set of client computer systems.
 6. The method of claim5, wherein the first attribute comprises at least one of: a processorspeed; an amount of memory; and a network bandwidth.
 7. The method ofclaim 5, wherein the second attribute comprises at least one of: networktraffic; a popularity; and a cost.
 8. The method of claim 1, whereinmerging the first set of updates into the second global version of thestatistical model comprises: using a set of weights associated with thefirst subset of the client computer systems to merge the first set ofupdates into the second global version.
 9. The method of claim 8,wherein the set of weights is associated with at least one of: a recencyof an update; a level of activity; a popularity; and a revenue.
 10. Themethod of claim 1, wherein the first set of updates is merged into thesecond global version based on a regularization parameter forpersonalizing the statistical model on each of the client computersystems and an optimization parameter for the set of client computersystems.
 11. An apparatus, comprising: one or more processors; andmemory storing instructions that, when executed by the one or moreprocessors, cause the apparatus to: transmit a first global version of astatistical model to a set of client computer systems; obtain, from afirst subset of the client computer systems, a first set of updates to afirst global version of a statistical model; merge the first set ofupdates into a second global version of the statistical model; andtransmit the second global version to the client computer systemsasynchronously from receiving a second set of updates to one or more ofthe first and second global versions from a second subset of the clientcomputer systems.
 12. The apparatus of claim 11, wherein the memoryfurther stores instructions that, when executed by the one or moreprocessors, cause the apparatus to: associate the first set of updateswith the second global version; use the associated first set of updatesto merge the second set of updates into a third global version of thestatistical model without merging the first set of updates into thethird global version; and transmit the third global version to the setof client computer systems.
 13. The apparatus of claim 12, wherein usingthe associated first set of updates to merge the second set of updatesinto the third global version without merging the first set of updatesinto the third global version comprises: identifying the first set ofupdates in a version identifier for the second global version; andomitting merging of the first set of updates into the third globalversion based on the identified first set of updates.
 14. The apparatusof claim 11, wherein the memory further stores instructions that, whenexecuted by the one or more processors, cause the apparatus to: selectthe set of client computer systems for distribution of the statisticalmodel from the apparatus based on a first attribute of the apparatus anda second attribute of the set of client computer systems.
 15. Theapparatus of claim 11, wherein merging the first set of updates into thesecond global version comprises: using a set of weights associated withthe first subset of the client computer systems to merge the first setof updates into the second global version.
 16. The apparatus of claim15, wherein the set of weights is associated with at least one of: arecency of an update; a level of activity; a popularity; and a revenue.17. The apparatus of claim 11, wherein the first set of updates aremerged into the second global version based on a regularizationparameter for personalizing the statistical model on each of the clientcomputer systems and an optimization parameter for the set of clientcomputer systems.
 18. A method, comprising: obtaining, from a server bya computer system, a first global version of a statistical model; usingdata on the computer system to produce a local version of thestatistical model from the first global version; and transmitting anupdate comprising a difference between the local version and the firstglobal version to the server asynchronously from receiving a secondglobal version of the statistical model from the server.
 19. The methodof claim 18, further comprising: obtaining a regularization parameterfor the computer system; producing the update based on theregularization parameter; and updating the local version with the secondglobal version based on the regularization parameter.
 20. The method ofclaim 19, further comprising: updating the regularization parameterbased on the data on the computer system.